AD-A219  807 


NAVAL  POSTGRADUATE  SCHOOL 
Monterey ,  California 


ISOLATION  OF  IMPORTANT  INPUT  FACTORS  IN  THE 
PERFORMANCE  OF  OPERATIONAL  PROPULSION  PLANT 
EXAMS  (OPPE)  AND  LIGHT  OFF  EXAMS  (LOE) 

FOR  ATLANTIC  FLEET  SHIPS 

by 

Thomas  D.  Nolen 

September  1989 

Thesis  Advisor: 

Dan  C.  Boger 

Approved  for  public  release;  distribution  is  unlimited 


SECURITY  CLASS  f  CATION  Oc  TH  S  PAG 


REPORT  DOCUMENTATION  PAGE 


Form  Approved 
OMB  No  0704  0188 


'a  REPORT  SECjR  Tv  CLASSIC  CANON 

Unclassified 


2a  SECURITY  CLASSiFlCAT  ON  AUTHORS 


2b  DECLASSIFICATION  '  DOWNGRADING  SCHEDULE 


A  PERFORMING  ORGANIZATION  REPORT  NUMBE R(S) 


ID  R£StRiC*  VE  VAR-  W 


3  DSTRBJ'-ON  AVAtA3."Y  OF  P£PO=- 

Approved  for  public  release; 
Distribution  is  unlimited 


5  MONITOR. NG  ORGAN. ZATION  REPORT  NUMBER'S, 


ba  NAME  OF  PERFORMING  ORGANIZATION  60  OFF.CE  SYMBOL 

(if  applicable) 

Naval  Postgraduate  School  55 


6c  ADDRESS  ( City ,  State,  and  ZIP  Code) 


7a  NAME  OP  MOVTORiNG  ORGAN  Z^T  ON 

Naval  Postgraduate  School 


7b  ADDRESS  (Ofy.  State  and  ZlPCode) 


Monterey,  California  93943-5000 


8a  NAME  OF  FUNDING 'SPONSORING 
ORGANIZATION 


Monterey,  California  93943-5000 


3d  GFmCc  S  t'MdU-  9  PROCUntMtN’  INSTRUMENT  iDENT|FiCAt,ON  NUMBt* 

(If  applicable) 


8c  ADDRESS  (City,  State,  and  ZIP  Code) 


10  SOURCE  Of  FUND;NG  NUMBERS 


PROGRAM  I  PRO,ECT 

element  no  I  NO 


WORK  UNIT 
ACCESSION  NO 


11  TITLE  (Include  Security  Classification)  ISOLATION  OF  IMPORTANT  INPUT  FACTORS  IN  THE 
PERFORMANCE  OF  OPERATIONAL  PROPULSION  PLANT  EXAMS  (OPPE)  AND  LIGHT  OFF 


L»J3 


12  PERSONAL  AJThORiS, 

NOLEN,  Thomas  D. 


13a  TYPE  OF  REPORT  r  3o  TIME  COVERED  14  DATE  OF  REPOR*  (real  Month  Day)  S  PAGE  CO*VV 

Master’s  thesis  I  from _ to _  1989  September  53 


16  supplementary  notation  1 ne  views  expressed  m  this  thesis  are  those  of  the 
author  and  do  not  reflect  the  official  policy  or  position  of  the 
Department  of  Defense  or  the  U.S.  Government 


17  COSATi  CODtS  I  18  ^SUBJECT  TERMS  ( Continue  on  reverse  it  necessary  and  identity  by  block  number) 

T5  |  GROUP  |  SUB-GROUP  j  OPPE,  LOE,  PEB ,  Multinomial  linear, 

regression.  Operational  Propulsion  Plant  Exams. 
Light  off  Exams. 


19\ABS'RACT  ( Continue  on  reverse  if  necessary  and  identify  by  block  number) 

^This  thesis  examines  the  relationships  between  independent  variables 
such  as  underway  time,  material  conditions,  and  personnel  manning  and 
dependent  variables  in  the  form  of  engineering  exam  scores.  Multinomial 
linear  regression  models  are  used  to  examine  these  relationships.  These 
efforts  met  with  limited  success.  The  percent  of  time  that  a  ship  spends 
underway  prior  to  an  OPPE  was  the  most  significant  of  any  independent 
variable  considered,  yet  efforts  to  model  the  effects  of  diminishing 
returns  were  unsuccessful.  Outship  OPPEs  failed  to  show  any  significant 
relationship  for  the  underway  independent  variables  examined,  but  they 
did  reveal  that  ships  which  file  a  greater  number  of  CASREPs  prior  to 
receiving  an  outchop  OPPE  increase  their  odds  of  receiving  a  favorable 
test  score.  Attempts  to  model  LOE  were  unsuccessful.  7.  ■*<16  (  _  ■ 


20  US-p-i.'uN  A2A  „  AB  _  -V  OF  AB  "PAT¬ 
CH  AC.  ASS  ;  f  D  -A.  V  □  SAVE  AS  ‘  □  :>*  f  iSt  as 


•i *.  a  /:  Or  S^OVj  8..:  1 

Dan  C.  Boaer 


uO  Form  14/3,  JUN  86  Prcv  us  t>d>tions  ar<  onsntr.ts  _ '  :  -  '  *  (  .  ■* S  ;  <  ’  •*.  •  ; 


J  ’  A  B  >  "  *  A  C  ’  S  i '  »*-  ,  •  v.  /• 

Unclassified 

(fCwb  &  .:<K. 

•  t  -  :  n  .  V:-5 

498-645-2607 

553o 

Prcv  JS  f'd'tinns  Art  nn$r>lrt*' 

S  'N  (•  1  ■; j-I.:  -d I  u-Aa;  : 


Unclassified 


\J  .  A. 


Approved  for  public  release,  distribution  is  unlimited 


ISOLATION  OF  IMPORTANT  INPUT  FACTORS  IN  THE 
PERFORMANCE  OF  OPERATIONAL  PROPULSION  PLANT  EXAMS  (OPPE) 
AND  LIGHT  OFF  EXAMS  (LOE)  FOR  ATLANTIC  FLEET  SHIPS 


by 


Thomas  D.  Nolen 
Lieutenant,  United  States  Navy 
B.S.  United  States  Naval  Academy,  1984 

Submitted  in  partial  fulfillment 
of  the  requirements  for  the  degree 


MASTER  OF  SCIENCE  OPERATIONS  RESEARCH 

NAVAL  POSTGRADUATE  SCHOOL 
September  1989 


Author: 


Approved  by 


Thomas  D.  Nolen 


ABSTRACT 


This  thesis  examines  the  relationships  between  independent 
variables  such  as  underway  time,  material  condition,  and 
personnel  manning  and  dependent  variables  in  the  form  of 
engineering  exam  scores.  Multinomial  linear  regression  models 
are  used  to  examine  these  relationships.  These  efforts  met 
with  limited  success.  The  percent  of  time  that  a  ship  spends 
underway  prior  to  an  OPPE  was  the  most  significant  of  any 
independent  variable  considered,  yet  efforts  to  model  the 
effects  of  diminishing  returns  were  unsuccessful.  Outchop 
OPPEs  failed  to  show  any  significant  relationship  for  the 
underway  independent  variables  examined,  but  they  did  reveal 
that  ships  which  file  a  greater  number  of  CASREPs  prior  to 
receiving  an  outchop  OPPE  increase  their  odds  of  receiving  a 
favorable  test  score.  Attempts  to  model  LOE  were 
unsuccessful . 
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I.  INTRODUCTION 


A.  BACKGROUND 

The  ability  of  a  Navy  ship  to  perform  its  peacetime  and 
wartime  missions  is  a  function  of  its  various  departments' 
(operations,  weapons,  engineering,  navigation,  and  supply) 
abilities  to  conduct  their  individual  tasks.  The  primary 
goals  of  the  engineering  department,  providing  propulsion, 
generating  electricity,  and  controlling  battle  damage,  are  an 
intricate  part  of  the  ability  of  the  ship  as  a  whole  to 
perform  its  mission.  Consequently,  the  measurement  of  a 
ship's  engineering  readiness  plays  a  role  in  the  overall 
assessment  of  the  ship's  ability  to  complete  its  mission. 
To  evaluate  the  engineering  department  of  each  conventionally 
powered  surface  ship,  the  Commander  in  Chief,  Atlantic  Fleet, 
established  the  Propulsion  Examining  Board. 

The  Atlantic  Fleet  Propulsion  Examining  Board  (PEB) 
consists  of  Naval  officers  with  shipboard  engineering 
experience.  Headed  by  a  post-command  Captain,  the  board  is 
divided  into  several  inspection  teams.  Each  examination  team 
consists  of  a  post-command  Commander,  as  well  as  other 
officers  with  engineering  experience  at  the  department  head 
and  division  officer  levels.  The  head  of  the  PEB  is  tasked 
with  inspecting  Atlantic  Fleet  ships  using  a  uniform  standard 
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of  engineering  readiness.  Since  the  board  operates  out  of  one 
office,  with  one  clear  set  of  standards,  exam  results  may  be 
viewed  as  a  good  aggregate  measure  of  a  ship's  engineering 
readiness.  The  PEB  conducts  two  basic  types  of  examinations. 
Operational  Propulsion  Plant  Examinations  (OPPE's)  and  Light 
Off  Examinations  (LOE's). 

The  primary  purpose  of  an  LOE  is  to  determine  the  ship's 
ability  to  safely  operate  the  engineering  plant  equipment. 
Additionally,  an  LOE  evaluates  the  ability  of  a  ship  to  combat 
a  flammable  liquid  fire  in  a  main  engineering  space.  LOE's 
are  typically  administered  after  an  idle  period  of  120  days 
or  more  of  the  ship's  engineering  plant.  Usually,  such  a 
period  of  inactivity  occurs  during  a  period  of  maintenance 
such  as  a  regular  overhaul  (ROH)  or  selected  restricted 
availability  (SRA)  .  An  LOE  is  also  administered  prior  to  the 
commissioning  of  a  ship. 

On  the  other  hand,  OPPE's  are  administered  to  a  ship  in 
an  operating  status.  The  PEB  conducts  an  OPPE  for  each  ship 
approximately  every  24  months.  While  an  OPPE  also  evaluates 
the  ship's  abilities  to  safely  operate  propulsion  plant 
equipment  and  to  combat  a  flammable  liquid  fire,  the  focus  of 
the  exam  is  somewhat  different.  The  PEB  examines  the  ability 
of  the  ship  to  man  a  three  section  underway  watchbill.  It 
also  determines  the  ability  of  the  ship  to  conduct  its  own 
underway  training  and  evaluation  using  in-house  evaluation 
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teams  known  as  the  Engineering  Casualty  Control  Training  Team 
(ECCTT)  and  the  Damage  Control  Training  Team  (DCTT) . 

The  OPPE  and  LOE  are  the  best  overall  measures  of  a  ship's 
engineering  readiness.  The  examination  criteria  are  the  most 
comprehensive  of  any  exam  administered  to  surface  units.  The 
PEB  compares  current  and  past  ship  engineering  procedures 
against  prescribed  fleet  and  type  commander  requirements. 
The  PEB  assigns  qualitative  grades  to  the  exam  results  for 
both  exams.  Exams  do  not  receive  quantitative  scores  on  a 
numeric  scale,  for  example,  zero  to  100.  Each  exam  receives 
an  overall  score  which  is  a  reflection  of  the  results  of 
various  subareas  such  as  the  ship's  material  status,  level  of 
knowledge,  training,  firefighting,  ability  to  operate 
equipment  safely,  and  the  like.  The  LOE ' s  are  evaluated  on 
a  pass-fail  basis,  while  OPPE's  receive  relative  scores  such 
as  above  average,  average,  below  average,  and  unsatisfactory. 

The  material  evaluation  of  the  ship's  engineering  plant 
includes  both  a  static  and  dynamic  inspection.  Prior  to  any 
other  major  portion  of  the  inspection,  the  PEB  evaluates  the 
physical  condition  of  the  equipment.  The  crew  provides  an 
initial  assessment  which  establishes  that  a  specified  minimum 
level  of  equipment  is  functioning.  The  PEB  then  requires 
selected  equipment  to  be  demonstrated  as  operational;  for 
example,  a  qualified  operator  may  be  required  to  start  a  fire 
pump  correctly  and  place  it  in  operation.  An  evolution  such 
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as  this  also  helps  PEB  to  assess  the  level  of  knowledge  of  the 
engineering  department. 

The  PEB  evaluates  the  level  of  knowledge  of  the  department 
through  the  successful  operation  of  plant  equipment,  as 
previously  discussed.  It  also  utilizes  oral  and  written 
exams.  The  written  exams  arrive  at  the  ship  just  prior  to  the 
exam.  Critical  watchstanders  also  undergo  a  formal  board  of 
oral  examination.  In  addition,  crew  members  are  also  subject 
to  questioning  by  PEB  members  at  any  point  during  the  exam. 

The  engineering  department's  operating  logs,  legal,  and 
administrative  records  are  also  inspected  for  proper  format 
and  recordkeeping.  The  administrative  review  helps  to 
establish  the  ship's  ability  to  implement  required,  safety- 
related  programs.  It  also  inspects  the  ability  of  the  ship 
to  conduct  requisite  watchstation  training  and  follow  up  with 
the  appropriate  documentation.  The  review  in  these  areas 
focuses  on  the  ability  of  the  ship  to  schedule  personnel  to 
attend  required  schools,  to  conduct  required  organizational 
training  onboard,  and  to  correctly  record  these  events  in  the 
required  manner.  The  PEB  also  inspects  the  ship's  operating 
and  legal  records.  This  inspection  provides  additional 
insight  into  the  material  status  of  the  equipment.  It  also 
demonstrates  the  ability  of  the  crew  to  identify  problem 
areas,  for  instance  an  out  of  tolerance  temperature  reading, 
and  to  take  corrective  action.  However,  the  operating  log 


inspection  is  but  one  technique  available  to  PEB  for  the 
evaluation  of  proper  casualty  control  procedures. 

Perhaps  the  most  intense  period  of  the  exam  occurs  during 
the  graded  casualty  control  drills.  During  these  drills,  the 
ship's  own  casualty  control  training  teams,  ECCTT  and  DCTT, 
conduct  casualty  drills  on  various  equipment.  The  PEB 
evaluates  the  ability  of  the  shipboard  team  to  impose, 
conduct,  and  evaluate  these  drills,  as  well  as  the  ability  of 
the  crew  to  effectively  handle  the  casualty.  The  PEB  members 
constantly  question  and  evaluate  every  level  of  watchstander  - 
-  from  the  commanding  officer  through  the  engineering  officer 
of  the  watch  to  the  lowest  ranking  fireman  on  duty  in  the 
engineering  spaces  —  to  determine  each  individual's  current 
assessment  of  the  casualty,  their  required  actions,  and 
subsequent  possible  effects.  The  majority  of  the  drills  focus 
solely  on  the  engineering  department,  but  one  area  of 
evaluation,  firefighting,  concerns  all  hands. 

The  PEB's  assessment  of  firefighting  focuses  on  the 
ability  of  the  ship  to  combat  a  major  fire  in  a  main 
engineering  space.  A  typical  scenario  involves  the  discovery 
of  a  major  flammable  liquid  leak  in  a  main  engineering  space. 
As  the  drill  advances,  the  casualty  becomes  more  complex, 
usually  progressing  into  a  main  space  fire.  While  engineering 
department  personnel  would  typically  combat  such  a  fire, 
additional  personnel  are  tasked  to  provide  backup  and 
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support.  The  PEB  will  typically  evaluate  the  qualifications 
and  performance  levels  of  these  personnel  as  well.  The  main 
space  fire  drill  is  one  of  the  most  difficult  phases  of  the 
entire  exam. 

The  PEB  administers  the  most  comprehensive  engineering 
examinations  for  surface  ships.  While  other  measures  of 
readiness,  such  as  casualty  reporting  procedures,  may  provide 
a  more  detailed  static  assessment,  PEB  provides  the  only 
uniform  inspecting  team  which  evaluates  the  current  status  and 
inspects  the  past  operating  procedures  in  an  attempt  to  ensure 
safe  engineering  plant  operation  in  the  future. 

B .  PURPOSE 

Since  PEB  administers  exams  on  a  fleet  wide  basis, 
examination  results  are  of  great  interest  to  the  other  levels 
of  the  chain  of  command,  such  as  squadron,  group,  and  type 
commander,  in  addition  to  the  ships  themselves.  While  all 
levels  strive  for  high  engineering  readiness  as  a  goal,  each 
attempts  to  specifically  aid  or  prepare  a  ship  for  an  upcoming 
exam.  Perhaps  the  best  example  is  that  of  the  Commander  Naval 
Surface  Force  Atlantic  (CNSL)  Engineering  Mobile  Training  Team 
(EMTT) .  This  team  conducts  periodic  and  preliminary 
inspection  visits  on  ships  in  an  effort  to  make  them  better 
prepared  for  an  OPPE  or  an  LOE.  The  squadron  or  type 
commander  may  also  schedule  an  additional  maintenance  period, 
or  additional  days  underway  for  training,  in  the  hope  of 
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achieving  a  higher  score.  In  fact,  CNSL  maintains  a  shipboard 
training  and  readiness  division,  N6,  which  is  tasked  with 
monitoring  ship  engineering  readiness  and  assisting  ships  in 
their  preparation  for  engineering  inspections. 

Both  the  type  commander  and  the  squadron  commander  support 
a  ship's  efforts  to  increase  its  score  on  an  OPPE  or  LOE. 
Both  staffs  contain  personnel  intimately  familiar  with 
shipboard  engineering  requirements  through  their  own 
experiences  at  the  senior  enlisted  and  officer  level.  With 
such  a  high  level  of  interest  focused  on  this  one  measure  of 
effectiveness,  the  examination  process  becomes  a  natural  topic 
for  academic  study.  The  Center  for  Naval  Analysis  (CNA)  has 
conducted  the  majority  of  previous  work  is  this  area. 

Linda  Cavalluzzo,  a  CNA  analyst  working  at  the  request  of 
CNSL,  performed  the  initial  study  to  interpret  possible 
critical  inputs  into  the  exam  scores  [Ret.  1].  It  focuses  on 
the  effect  of  an  increased  number  of  days  underway  for  CNSL 
ships  receiving  an  OPPE.  Cavalluzzo  uses  a  pass/fail 
criterion  as  her  measure  of  effectiveness  (MOE) .  While  her 
results  are  promising  (i.e.,  thev  indicate  an  increased  chance 
of  receiving  a  passing  grade  for  tnose  ships  with  a  greater 
number  of  underway  days) ,  her  study  is  limited  to  a  small 
sample  of  ships  (22)  aver  a  short  period  of  time.  The  brevity 
of  the  analysis  and  small  number  of  observations  are  no  doubt 
the  fallout  of  a  timely  requirement  for  the  results. 


Alan  Marcus  et  al.  also  attempt  to  evaluate  the 
relationships  between  OPTEMPO,  the  number  of  days  a  ship 
spends  underway,  and  readiness  [Ref.  2].  The  study  is  more 
comprehensive  in  nature,  covering  a  greater  number  of  ships 
(134)  over  a  longer  period  of  time  (1982-1985).  The  study 
uses  the  results  of  a  selected  exercise  (SELEX)  in  navigation, 
the  ability  of  individual  shipboard  departments  to  win 
competitive  awards,  as  well  as  OPPE  results  as  the  primary 
MOEs.  Marcus  also  chooses  the  pass/fail  criterion  as  the 
primary  OPPE  MOE,  thus  combining  a  possible  four  or  five 
choices  into  two  distinct  categories.  He  also  focuses  the 
majority  of  his  analysis  on  one  critical  input  factor  — 
OPTEMPO.  Marcus  utilizes  a  similar  rationale  for  review  as 
did  Cavalluzzo,  yet  he  concludes  that  the  pass  rate  for  an 
OPPE  had  no  association  with  OPTEMPO.  This  sets  the  stage  for 
the  current  analysis. 

This  thesis  builds  on  the  work  of  Marcus  and  Cavalluzzo 
in  an  attempt  to  discover  a  resources-to-readiness 
relationship  between  various  critical  input  factors  and  their 
effect  on  OPPE  and  LOE  exam  scores.  Previous  work  has  focused 
primarily  on  OPTEMPO  as  an  important  factor.  This  thesis  will 
consider  a  broader  scope  of  inputs,  such  as  various 
measurements  of  material  readiness  as  well  as  the  level  of 
manning.  It  will  also  evaluate  the  level  of  detail  provided 
by  the  exam  results  —  resisting  the  temptation  to  utilize  a 
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simple  pass/fail  criterion  —  and  will  rely  instead  on  the 
greater  qualitative  distinction  provided  by  the  exam  scores. 

By  broadening  the  scope  of  study  in  the  number  of  exams 
reviewed,  possible  input  factors  considered,  and  the 
assessment  of  output  measures,  this  thesis  seeks  to  determine 
relationships  that  decision  makers  may  use  in  assessing 
various  input  factors  and  their  effects  on  ship  exam  scores. 
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II 


DATA  AND  80URCES 


A.  INPUT  DATA 

As  with  the  previous  studies  this  effort  utilizes  data 
from  a  CNA  database.  The  database  contains  data  on  152 
surface  ships  from  fiscal  year  FY  1978  through  FY  1988.  The 
variables  reflect  monthly  measures  of  their  respective 
quantities,  with  the  exception  of  several  manpower  variables, 
which  due  to  CNA  computation  reflect  90  or  180  day  values. 
The  data  originate  from  a  number  of  different  sources,  most 
of  which  are  primary  monitoring  or  decision  making  tools  for 
the  Navy  or  Department  of  Defense.  Initially  compiled  in 
response  to  a  CNA  effort  concerning  the  factors  affecting  ship 
material  condition,  the  variables  cover  a  wide  range  of 
topics.  Reference  3  describes  these  sources  in  depth.  In 
part,  the  sources  include  the  following: 

-  Ship  Employment  History  (SEH) 

-  Officer  Master  Files 

-  Defense  Manpower  Data  Center's  Enlisted  Personnel 
Unit  Identification  Code  File 

-  Enlisted  Master  Record  Files 

-  Enlisted  Billet  Files 

-  Board  of  Inspection  and  Survey 

-  3-M  Maintenance  Data  System 
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3-M  Parts  Records 


-  Casualty  Reporting  System  (CASREPs) 

-  Ship  Fuel  and  Steaming  Hours  Data 

In  general,  the  data  fall  into  one  of  three  basic 
categories  based  on  the  quantities  which  they  describe: 
material  condition  of  the  ship,  manpower,  or  underway  time. 

The  CNA  database  contains  several  useful  measures  of  each 
ship's  underway  time.  The  variables  which  describe  the 
percent  underway  and  the  percent  in  port  each  month  are  the 
most  simple  and  direct  measure.  When  they  are  added  together, 
they  sum  to  100.00.  Another  variable  contains  the  number  of 
hours  steaming  underway  in  a  month.  Its  companion  contains 
the  number  of  hours  steaming  not  underway  in  a  month.  This 
measure  allows  analysis  in  the  scenario  where  a  ship  operates 
the  engineering  plant  tied  to  the  pier  or  at  anchorage.  Other 
underway  variables  include  the  percent  deployed,  the  percent 
long  deployment  (i.e.,  when  a  ship  is  on  duty  in  the  Western 
Pacific,  Indian  Ocean,  or  Persian  Gulf) ,  and  the  percent  of 
extended  operations  (greater  than  eight  weeks  away  from 
homeport) . 

Several  related  measures  of  manpower  are  also  present  in 
the  database.  Manpower  variables  divide  easily  into  two 
different  types:  those  which  describe  the  current  level  of 
manning  relative  to  the  prescribed  level  and  those  which 
measure  the  rate  of  crew  turnover. 
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The  measures  of  crew  turnover  are  for  90  (one  quarter)  and 
180  day  (two  quarters,  or  six  months)  periods.  Values 
available  include  the  percent  new  crew  overall,  which  applies 
to  all  enlisted  personnel  job  specialties,  or  ratings,  and  the 
percent  new  crew  in  the  engineering  job  ratings. 

The  variables  which  describe  the  manning  level  relative 
to  requirements  also  display  values  for  both  the  entire  crew 
and  engineering  ratings.  The  CNA  data  use  two  weights  in  the 
computation  of  these  ratios.  The  first  weight  utilizes  only 
the  difference  in  pay  among  enlisted  personnel.  The  second 
attempts  to  capture  differences  in  the  productivity  of 
enlisted  personnel  based  on  their  paygrade.  The  weighting 
scheme  is  the  result  of  CNA  follow  up  work  on  a  Rand 
Corporation  study.  Appendix  D  of  Reference  3  contains 
additional  details.  The  numerator  values  of  these  ratios,  the 
manning  levels,  are  also  available  for  study. 

Manning  values  are  subdivided  into  three  paygrade  groups. 
Manning  to  requirements  measures  exist  for  paygrades  of  junior 
personnel  (E1-E3) ,  mid-grade  petty  officers  (E4-E6) ,  and 
senior  petty  officers  (E7-E9)  for  the  overall  crew.  A  measure 
of  this  ratio  is  also  available  for  mid-grade  engineering 
petty  officers  (E4-E6) . 

The  final  input  variables  available  for  study  are  those 
which  describe  the  material  condition  of  the  ship.  All  of 
these  measures  utilize  Casualty  Reporting  (CASREP)  data.  A 
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CASREP  is  a  message  sent  from  the  Commanding  Officer  of  a  ship 
to  his  administrative  and  operational  commanders  describing 
equipment  which  is  not  functioning  as  designed.  These 
messages  document  discrepancies  which  cannot  be  repaired 
within  48  hours  after  discovery.  Three  categories  exist  to 
describe  the  degree  of  degradation,  C2 ,  C3,  and  C4,  in 
increasing  order  of  seriousness. 

All  of  the  data  present  describing  material  readiness 
prior  to  exams  in  the  CNA  database  utilize  measures  derived 
from  CASREPs.  All  values  measure  monthly  events.  The  first 
variable  measures  the  number  of  CASREPs  which  begin  in  a  given 
month  by  degree  of  degradation.  One  value  also  exists  which 
describes  the  number  of  C3  and  C4  CASREPs  which  begin  in  a 
given  month;  it  is  the  sum  of  the  individual  measures.  A 
variable  also  exists  which  describes  the  number  of  days  a  ship 
spends  in  a  C3  status  in  the  month.  For  example,  if  a  ship 
has  two  C3  CASREPs  for  a  30  day  period,  this  value  would  be 
60.  Again,  an  aggregate  measure  exists  for  the  C3  and  C4 
status  combined.  Finally,  variables  exist  to  describe  the 
number  of  days  that  a  ship  is  free  of  CASREPs.  These  values 
range  from  0  to  31,  with  the  exception  of  the  aggregate 
measure,  Days  Free  of  C3/C4  CASREPs.  This  measure  has  values 
in  the  range  from  0  to  62. 

At  least  one  caveat  is  important  to  note  in  the  material 
measures  at  this  juncture.  The  CASREP  data,  while  accurately 
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describing  the  material  condition  of  the  ship,  includes 
casualties  from  other  areas  of  the  ship.  Consequently,  a 
casualty  from  the  combat  systems  department  will  affect 
variable  values,  yet  may  be  insignificant  in  the  ability  of 
the  ship  to  pass  an  engineering  exam. 

One  shortfall  exists  in  the  overall  database,  a  result  of 
the  past  focus  of  CNA  studies.  In  general,  auxiliary  ships 
(such  as  ammunition  ships,  oilers,  and  food  carrying  ships) 
and  amphibious  ships  (such  as  helicopter  assault  ships  and 
tank  landing  ships)  lack  a  complete  set  of  input  variables  for 
study.  This  is  the  result  of  CNA  studies  which  center  on 
combatants.  However,  due  to  several  CNA  projects  concerning 
manpower,  the  majority  of  the  manpower-related  input  variables 
are  present  for  all  ships.  The  initial  effect  of  this 
discovery  requires  more  careful  procedures  when  processing  the 
data,  and  the  ultimate  impact  is  to  restrict  the  range  of 
ships  to  which  possible  conclusions  apply. 

In  summary,  the  CNA  database  contains  accurate  measures 
which  have  previously  been  used  for  modeling  and  analysis. 
While  the  data  were  initially  used  for  a  CNA  study,  they  have 
been  expanded  and  updated  to  cover  additional  areas  of 
interest. 
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B.  OUTPUT  DATA 


Output  data  are  primary  source  data.  They  come  directly 
from  the  PEB.  The  data  describe  the  results  of  every  type  of 
exam  administered  by  the  PEB.  As  previously  discussed,  the 
PEB  gives  two  basic  types  of  exams,  OPPEs  and  LOEs.  Their 
database  distinguishes  some  differences  among  these  two  types, 
however.  For  example,  the  results  differentiate  between  the 
exams  and  reexams.  Since  the  majority  of  reexaminations  occur 
shortly  after  the  initial  evaluation  date,  the  input  variables 
are  not  noticeably  different.  Also,  the  PEB  usually 
reinspects  only  those  failing  areas  of  the  initial  exam.  For 
these  reasons,  reexams  are  not  reviewed  in  this  study.  It 
also  distinguishes  between  regular  OPPEs  and  OPPEs  given  on 
the  way  home  from  a  major  deployment.  The  latter  type  of 
exam  is  called  an  outchop  OPPE.  The  data  also  break  out  exams 
administered  by  the  Atlantic  Fleet  Training  Group  (FTG)  in 
Guantonomo  Bay,  Cuba.  Since  this  specific  type  of  exam  is  no 
longer  administered,  it  is  not  considered  in  this  review. 

Basically,  the  PEB  data  consist  of  the  exam  results,  both 
the  overall  score  and  every  subarea.  It  also  contains 
ancillary  information  such  as  the  ship's  homeport, 
administrative  squadron  assignment,  etc.  As  previously 
discussed,  the  PEB  assigns  qualitative  grades  such  as 
unsatisfactory,  below  average,  average,  and  above  average  to 
OPPE  exam  results,  and  evaluates  LOEs  on  a  pass/fail  basis. 
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The  PEB  also  records  incomplete  exams  and  the  portions  of 
exams  which  are  not  given.  This  policy  results  in  a 
comprehensive  data  collection  mechanism.  While  the  majority 
of  the  PEB  data  concern  output  measures,  three  input  variables 
are  also  present. 

The  PEB  database  denotes  the  experience  level  of  the 
engineering  officer  by  showing  the  number  of  months  which  he 
has  been  in  his  current  job.  It  reflects  the  experience  level 
of  the  Commanding  Officer  (CO)  in  the  same  manner.  The  PEB 
also  records  if  the  CO  was  an  engineer  when  he  served  as  a 
department  head.  All  of  these  factors  contribute  to  the  level 
of  managerial  expertise  on  board  prior  to  the  exam. 

In  summary,  the  PEB  database  contains  the  exam  results 
collected  by  the  examination  teams.  With  a  Commander 
verifying  the  results  prior  to  release,  and  one  officer 
managing  the  database,  the  data  are  very  accurate.  The  PEB 
data  were  matched  to  the  CNA  data  and  uploaded  into  the  Naval 
Postgraduate  School  (NPS)  mainframe  computer  for  processing. 
A  discussion  of  the  methodology  follows. 
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III.  METHODOLOGY 


As  stated  in  the  introduction,  this  thesis  examines  the 
resource-to-readiness  relationships  between  input  factors,  or 
independent  variables,  and  engineering  exam  scores,  or 
dependent  variables.  Traditional  approaches  which  search  for 
a  relationship  between  independent  and  dependent  variables 
often  utilize  simple  linear  regression.  However,  due  to  the 
structure  of  the  problem  under  study,  and  the  assumptions  of 
linear  regression,  this  technique  is  inappropriate.  A  related 
procedure,  called  logistic  regression,  is  the  effective 
analytical  tool  for  the  job.  The  following  section  will 
demonstrate  the  suitability  of  logistic  regression,  after 
describing  the  shortfalls  of  linear  regression. 

Consider  a  simple  linear  regression  model  of  an  exam  which 
assigns  pass/fail  scores.  Linear  regression  uses  the  equation 

y,-  =  a  +  bXj  +  e{ 

for  i  =  1,...,N,  where  y ■  is  a  random  variable  which  takes  on 
the  value  of  1  if  ship  i  passes  the  exam  and  0  if  it  fails, 
a  and  b  are  unknown  parameters,  x(  is  the  input  factor  under 
consideration,  and  ei  is  a  random  disturbance  which  is  assumed 
to  be  normally  distributed  with  a  mean  of  zero  and  a  variance 
of  a 2  for  all  i.  Let  P(  be  the  probability  that  y-  =  1,  and 
let  (1  -  Pj )  be  the  probability  that  y-  =  0.  Note  that  if 
E[ef]  =  0,  then  E[y(]  =  P;  =  a  +  bxs .  This  approach 
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immediately  generates  a  number  of  problems.  Note  that  while 
y;  is  restricted  to  0  or  1,  the  systematic  portion  of  the 
right  hand  side  of  the  equation  may  take  on  any  value. 
Consequently,  e(  can  have  only  two  values,  -(a  +  bx()  and  1- 
(a  +  bxt)  .  It  assumes  these  values  with  probabilities  of  1  - 
(a  +  bXj)  and  -(a  +  bXj)  ,  respectively.  Since  a  +  bx(  can  have 
values  greater  than  one  or  less  than  zero,  P;  can  also  be  less 
than  0  or  greater  than  1.  These  glaring  contradictions  are 
further  compounded  by  the  variance  of  e-.  The  traditional 
assumption  that  Var(e^)  =  o 2  for  all  i  no  longer  applies. 
Instead,  the  Bernoulli  character  of  yi  requires  that  the 
variance  for  e,  become  (a  +  bxf)  (1  -  a  -  bx().  Notice  that 
the  variance  of  e(  now  depends  on  i.  This  violates  of  one  of 
the  basic  assumptions  of  simple  linear  regression  that  the 
variance  of  ef  is  the  same  for  all  i.  Finally,  computations 
for  E  (y j  |  x, )  may  result  in  values  outside  of  the  specified 
range  of  [0,1].  Judge  et  al.  [Ref. 5]  discuss  these  and  other 
shortfalls  of  using  linear  regression  to  model  this  scenario. 

Since  the  simple  linear  model  fails  to  correctly  depict 
the  basic  pass/fail  exam  outcome,  the  binary  logit  is 
considered  [Ref. 6].  The  binary  logit  model  utilizes  a  linear 
regression  model  of  the  following  form 

S 

y.*  =  B„+  Y  Bx  +u 

j  i  o  *->  j  i  j  i 

j  =  l 
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-nere  yf  is  not  observed.  It  is  called  a  "latent"  variable. 
The  value  actually  observed  is  a  dummy  variable  y(  where 

1  if  y*  >  0 

y«  = 

0  otherwise. 

The  term  j  represents  the  value  of  the  jth  independent 
variable  for  the  ith  observation.  The  vector  Bj ,  where 
j=l,...,S,  is  a  vector  of  coefficients  assigning  various 
weights  to  the  input  variables.  Note  that  in  this  model, 
unlike  the  previous  one,  Efyjxj]  is  not  B0  +  BjXjj,  rather 
these  terms  represent  the  expected  value  of  the  latent 
variable  y*.  The  previous  two  relationships  allow  us  to  state 

S 

=  Prob(yj  =  1  )  =  Prob [  u(  >  -(  B0  +  EB-x^  )  ] 

j  =  l 

S 

=  1  -  F[  -(  B0  +  E  BjXjj.)  ] 

j  =  l 

where  F  is  the  cumulative  distribution  of  u.  Using  the 
assumption  that  the  distribution  of  u  is  symmetric  (i.e.,  1- 
F ( -2 )  =  F ( Z ) ) ,  the  equation  takes  the  form 

S 

Ps  =  F(  B0  +  E  BjXjj  ). 

j  =  l 

The  functional  form  of  this  equation  depends  on  the 
assumptions  concerning  the  error  term  u.  If  the  cumulative 
distribution  of  u(  is  logistic,  then 
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discrete  output  is  possible.  An  extension  of  the  previous 
model  is  an  appropriate  choice. 

The  OPPE  results  consist  of  a  qualitative  response  in  the 
form  of  discrete  scores.  A  Rand  Corporation  study  [Ref.  7], 
discusses  such  a  structure,  where  the  response  variables  are 
categorical.  Let  y1  denote  an  assigned  score  for  ship  i, 
where  i  =  1,...,N.  Then  let 

Pu*  =  P{Yi  =  aj ) 

where  the  response  variable,  exam  score,  for  the  ith  ship  can 

assume  Q  values,  a1,...,aQ.  Note  that  . ®  P^*  =  1.  This 

requires  that  the  i  ship  must,  in  fact,  receive  one  of  the 

possible  scores.  The  response  variables  are  related  to  the 

input,  or  stimulus,  variables  by  the  standardized  multivariate 

logistic  CDF,  which  is  defined  by 

1 

F  ( t1  ,  .  .  .  ,  tn)  =  -  ,  -00  <  tj  <  00. 

n 

1  +  exp  (  -tj  ) 

j=l 

Defining  tj  in  terms  of  z,  and  applying  this  distribution  to 
the  proposed  model  yields 

.  exp(  Zjj  ) 

Pjj  =  /  i  =  l,...,N;  j  =  l,...,Q. 

Q 

exp(  zjk  ) 

k=l 
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Define  =  x^Bj,  where  x;  is  a  S  x  1  vector  of  input 
characteristics  for  ship  i.  B.  is  the  vector,  S  x  1  of  the 
coefficient  weights  to  be  estimated.  The  coefficients  may  be 
estimated  via  a  weighted  least  squares  technique  or  a  maximum 
likelihood  estimation  (MLE)  technique.  Until  recently,  the 
former  vas  utilized.  This  is  primarily  the  result  of  the  time 
required  to  implement  the  latter  on  a  computer;  however,  the 
software  package  which  supports  this  thesis,  Time  Series 
Processor  (TSP) ,  uses  MLE  techniques. 

Once  the  computer  generates  an  estimate  for  the 
coefficient,  it  is  time  to  assess  its  significance.  A  "t- 
test"  is  conducted  to  determine  if  the  given  estimate  is 
significantly  different  from  zero,  implying  a  relationship 
between  the  input  factor  and  exam  score.  Since  the 
coefficient  estimates  are  MLE's  and  the  estimated  standard 
errors  are  the  square  roots  of  the  diagonal  elements  of  the 
asymptotic  variance-covariance  matrix,  which  is  derived  from 
the  Cramer-Rao  lower  bound,  this  is  an  asymptotic  t-test.  The 
value  of  the  t-statistic  is  the  estimate  of  the  coefficient 
divided  by  the  standard  error  [Ref.  6].  In  most  cases,  the 
sign  of  the  coefficient  is  known  prior  to  computation.  For 
instance,  an  increase  in  the  number  of  days  free  of  CASREPs 
would  have  a  positive  impact  on  the  exam  score.  Consequently, 
a  one  sided  "t-test"  is  performed;  otherwise,  when  the  sign 
of  the  estimated  coefficient  is  unknown,  a  two-sided  test  is 
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used.  Throughout  the  course  of  analysis,  the  actual  value  of 
the  t-statistic  will  be  presented  to  avoid  disputes  concerning 
the  appropriate  level  of  significance. 
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IV.  PROPOSED  MODELS 


As  discussed  during  the  review  of  the  data,  available 
input  factors  fall  into  one  of  three  broad  categories: 
manpower,  underway  time,  or  material  condition.  The  data 
collection  techniques  used  in  assembling  the  database  result 
in  the  variables  having  a  high  degree  of  collinearity  within 
each  category.  Consider,  for  example,  those  variables 
measuring  a  ship's  material  condition.  All  of  the  measures 
are  based  on  the  CASREP  system.  The  number  of  days  a  ship 
spends  in  a  C3  status  is  simply  the  number  of  days  in  the 
month  minus  the  number  of  days  it  spends  free  of  C 3  CASREPs. 
In  a  similar  manner,  the  number  of  CASREPs  which  begin  in  a 
month's  time  influences  the  number  of  days  a  ship  spends  in 
a  casualty  status.  Consequently,  it  is  important  to  determine 
which  variable  within  each  category  makes  the  greatest  impact 
on  the  exam  score. 

The  simplest  way  to  approximate  which  variable  has  the 
greatest  significance  is  to  conduct  several  univariate  runs. 
The  sign  of  the  estimated  coefficient  is  not  the  critical 
issue,  since  it  is  usually  known  prior  to  the  run.  The 
magnitude  of  the  estimate  and  the  associated  significance  is 
what  counts.  A  value  of  zero  for  the  coefficient  would  imply 
no  relationship  between  the  independent  and  dependent 
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variables.  TSP  presents  the  estimated  coefficient,  standard 
error,  and  t-statistic  for  review  on  the  print  out.  The  value 
of  the  t-statistic  is  the  key  to  evaluating  the  importance  of 
the  estimate. 

Once  the  most  significant  variable  in  each  category  is 
determined,  multivariate  models  may  be  approximated  using  the 
most  significant  variables  from  the  univariate  analysis. 
The  OPPE  scores  are  a  function  of  the  underway  time,  manpower 
status,  and  material  condition.  These  values  are  used  by  TSP 
in  the  x  vector,  as  discussed  in  the  previous  section.  The 
B  vector  contains  the  values  of  the  estimated  coefficients. 

The  LOE  model  uses  the  multinomial  logit  formulation  as 
well,  since  the  multinomial  reduces  to  the  binomial  case  when 
there  are  only  two  possible  choices.  The  LOE  scores  are  a 
function  of  both  the  manpower  status  and  material  condition 
of  the  test  ship.  Since  LOE's  occur  after  a  ship  has  not 
been  underway  for  at  least  120  days,  underway  time  is  not 
considered  for  this  model.  The  ability  of  a  ship's  material 
status  to  influence  LOE  scores  also  becomes  a  concern  at  this 
point.  Recall  that  all  material  measures  are  based  on  the 
CASREP  system.  When  a  ship  enters  an  overhaul  period,  the 
reporting  procedures  for  casualties  change.  This  allows 
shipyard  workers  to  repair  and  replace  equipment  without  a 
CASREP  being  filed.  In  view  of  this  new  concern  about  the 
ability  of  material  variables  taken  in  combination  with 
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earlier  remarks  concerning  the  utility  of  the  material 
variable  as  a  whole,  the  value  of  this  measure  is  in  some 
doubt.  But  since  overhaul  periods  comprise  only  one  type  of 
event  which  precedes  an  LOE,  the  CASREP  measure  may  still  have 
some  validity. 
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V.  ANALYSIS 


A.  UNIVARIATE  MODELS 

TSP  presents  estimated  coefficients  for  review  in  tabular 
form.  Since  the  software  normalizes  the  estimated 
coefficients,  the  value  for  the  lowest  coefficient  is  set  to 
zero.  For  the  LOE,  where  the  exams  are  reported  as  a  pass  or 
fail,  the  effect  is  to  estimate  the  coefficients  reflecting 
the  constant  term  and  the  log  odds  of  passing  the  exam.  For 
the  OPPE,  the  coefficient  for  the  log  odds  of  receiving  a 
failing  score  is  set  to  zero.  Since  the  focus  of  this  thesis 
is  to  determine  important  input  factors  which  result  in  ships 
successfully  completing  these  exams,  this  development  is  not 
a  significant  shortcoming. 

Recall  that  the  majority  of  the  data  are  available  in 
monthly  values.  Runs  were  conducted  varying  the  number  of 
months  in  the  independent  value  from  one  to  six.  Usually, 
when  the  three  months  or  six  months  previous  to  the  month  in 
which  the  exam  was  given  were  used  as  the  input  value,  the 
most  significant  results  were  achieved.  Consequently,  one  or 
two  quarters  prior  to  the  month  of  the  exam  is  used  most  often 
for  the  independent  variable  values.  This  is  also  in  keeping 
with  the  planning  and  scheduling  timeframes  utilized  by  the 
Navy . 
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1.  Material  Variables 

As  previously  discussed,  univariate  runs  were 
conducted  using  the  LOE,  OPPE,  and  outchop  OPPE  as  output 
measures  across  all  three  variable  types  to  determine  which 
variable  from  each  of  the  three  categories  is  the  most 
significant.  Material  variables  were  considered  first.  When 
examining  the  effects  of  CASREP  status  on  OPPE  scores,  it  is 
logical  to  assume  that  those  values  which  measure  more  severe 
degradations  would  have  a  greater  impact  on  the  examination 
score.  Univariate  runs  using  material  measures  of  readiness 
support  this  assumption.  The  Days  Free  of  C3/C4  CASREPs  in 
the  quarter  prior  to  the  month  of  the  exam  (DF34-90)  is  the 
most  significant  material  variable  for  the  regular  OPPE.  As 
shown  in  Table  1,  all  of  the  signs  of  the  slope  coefficients 
are  positive  as  anticipated.  This  implies  that  more  days  free 
of  C3/C4  CASREPs  increase  the  log  odds  of  receiving  a  passing 
score  (i.e.,  below  average,  average,  or  above  average).  The 
estimated  slope  coefficient  for  those  ships  receiving  above 
average  scores  is  greater  than  the  other  estimates.  Its  t- 
statistic  value  is  also  the  greatest,  giving  it  a  level  of 
significance  of  less  than  0.05.  This  implies  that  there  is 
less  than  a  five  percent  chance  of  error  when  stating  that  of 
the  ships  receiving  an  OPPE,  an  increase  in  the  number  of  days 
free  of  C3/C4  CASREPs  increases  the  log  odds  of  attaining  that 
score.  The  other  estimated  coefficients  have  a  level  of 
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TABLE  l.  OPPE  vs.  DF34-90  (Degrees  of  Freedom  =  277) 


Parameter 


Estimate 


T-Statistic 


Below  Average 

Constant 

DF34-90 

Average 

Constant 

DF34-90 

Above  Average 

Constant 

DF34-90 


-0.7512428 

0.006690118 


-0.1199808 

0.006795538 


-3.120854 

0.02949016 


-1.083841 

0.6529236 


-0.2061219 

0.7839620 


-2.511097 

1.789323* 


♦significant  at  the  .05  level 


significance  greater  than  0.20,  but  they  are  not  significant 
at  the  0.05  level. 

Ships  receive  outchop  OPPEs  at  the  end  of  the 
deployment  enroute  to  homeport.  A  ship  has  usually  been  away 
from  homeport  for  a  period  of  approximately  six  months.  While 
the  Navy  endeavors  to  give  increased  support  to  deployed 
units,  the  maintenance  patterns  differ  from  a  non-deployed 
status.  For  instance,  parts  support  is  more  difficult.  While 
stores  ships  carry  spare  parts  for  deployed  ships,  their 
capacities  are  limited.  Also,  when  a  ship  is  away  from  its 
homeport  on  deployment,  it  does  not  have  access  to  the  local 
Navy  Supply  Center  or  local  civilian  suppliers.  Consequently, 
shipboard  patterns  for  CASREPs  change  during  a  deployment, 
perhaps  in  an  attempt  to  prompt  the  supply  system  for  needed 
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parts.  This  causes  the  most  important  material  measure  to  be 
different  for  outchop  OPPEs  than  for  regular  OPPEs.  The  most 
significant  material  measure  affecting  outchop  OPPE  scores  is 
the  number  of  C4  CASREPs  which  begin  in  each  month  for  the  six 
months  prior  to  the  month  of  the  exam  (BC4-180) .  As  shown  in 
Table  2,  the  sign  of  the  estimated  slope  coefficient  is 
certainly  not  as  expected.  In  short,  it  says  that  ships  which 

TABLE  2.  OUTCHOP  OPPE  VS.  BC4-180 
(Degrees  of  Freedom  =  72) 


Parameter 


Estimate 


T-Statistic 


Below  Average 

Constant  -1.4572926 

BC4-180  0.4040139 


-3.097871 

2.455777** 


Average 

Constant  -0.6774675 

BC4-180  0.3070861 


-1.820998 

2.001085* 


Above  Average 

Constant  -1.788364 

BC4-180  0.4261859 


-3.392287 

2.482655** 


*  Significant  at  the  0.05  level 
**  Significant  at  the  0.025  level 


file  more  C4  CASREPS  in  the  six  months  prior  to  the  month  of 
the  exam  have  an  increase  in  the  log  odds  of  receiving  a 
passing  score.  This  shipboard  strategy  is  not  unexpected  in 
general,  since  it  is  sometimes  employed  in  an  attempt  to 
expedite  parts  or  assistance  prior  to  the  start  of  a 


30 


deployment;  however,  it  is  surprising  that  it  surfaces  in  this 
instance  which  occurs  near  the  end  of  a  deployment.  The  t- 
statistic  value  is  significant  at  the  0.05  level,  and  the 
magnitude  of  the  estimated  coefficients  are  the  greatest  seen 
thus  far. 

Material  measures  also  influence  the  LOE  scores. 
Recall  that  ships  receiving  an  LOE  either  pass  or  fail  the 
exam.  The  LOEs  are  affected  most  by  the  number  of  days  of 
C3/C4  CASREPs  in  the  two  quarters  prior  to  the  month  of  the 
exam  (CD34-180)  ,  as  shown  in  Table  3.  The  sign  of  the 
coefficient  is  negative  as  anticipated,  indicating  that  an 
increase  in  the  number  of  days  of  C3/C4  CASREPs  has  a  negative 


TABLE  3.  LOE  vs.  CD34-180  (Degrees  of  Freedom  =  96) 


* 

Parameter 

Estimate  T-Statistic 

Passing 

Constant 

CD34-180 

2.794936  1.265409 

-0.05296375  -1.204150* 

*  Significant  at  the  0.10  level 

impact  on  achieving  a  passing  score.  The  magnitude  of  the 
estimated  coefficient  and  its  level  of  significance  are  less 
than  for  previous  input  factors  considered. 

2.  Underway  Time  Variables 

Those  variables  which  measure  the  amount  of  time  a 
ship  spends  underway  are  the  next  category  for  discussion. 
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Recall  that  since  an  LOE  is  administered  after  a  prolonged 
inport  status,  it  is  not  discussed  here.  As  shown  in  Table 
4,  the  variable  which  measures  the  percent  of  time  a  ship  is 
underway  in  the  six  months  prior  to  the  month  of  the  exam 
(PRUW-180)  has  the  greatest  impact  on  the  OPPE  exam  scores. 
Unlike  the  CASREP  variables,  which  were  the  most  significant 
for  the  quarter  before  the  exam,  it  appears  that  underway  time 
TABLE  4.  OPPE  vs.  PRUW-180  (Degrees  of  Freedom  =  277) 

Parameter  Estimate  T-Statistic 

Below  Average 

Constant  -1.430346  -2.143820 

PRUW-180  0.006204640  1.847222* 

Average 

Constant  -1.186394  -2.029214 

PRUW-180  0.008172133  2.762979** 

Above  Average 

Constant  -3.429899  -3.416571 

PRUW-180  0.01122343  2.493490** 


*  Significant  at  the  0.05  level 
**  Significant  at  the  0.01  level 

has  the  most  impact  for  the  six  months,  or  two  quarters,  prior 
to  the  exam.  Again,  the  signs  of  the  estimated  slope 
coefficients  are  positive  as  anticipated,  implying  that  a 
higher  percentage  of  time  underway  is  beneficial  in  the  six 
months  prior  to  the  month  in  which  the  exam  is  given.  The  t- 
statistic  values  for  the  estimated  coefficients  are  all 


significant.  The  least  significant  has  a  level  of 
significance  of  less  than  0.05. 

The  percent  of  time  spent  underway  in  the  three  months 
prior  to  the  exam  (PRUW-90)  is  the  most  significant  underway 
measure  for  outchop  OPPEs.  The  signs  of  the  estimated  slope 
coefficients  in  Table  5  are  positive,  as  anticipated.  Both 
the  coefficient  and  the  t-statistic  value  are  the  greatest  for 
the  average  test  score,  implying  that  an  increase  in  the 
percent  of  time  underway  had  the  greatest  impact  in  achieving 
this  score.  The  lowest  t-statistic  value  occurs  for  the  above 
average  score,  and  with  a  level  of  significance  of  less  than 
0.10. 

TABLE  5.  OUTCHOP  OPPE  vs.  PRUW-90  (Degrees  of  Freedom  =  71) 


Parameter 


Estimate 


T-Statistic 


Below  Average 

Constant 

PRUW-90 

Average 

Constant 

PRUW-90 

Above  Average 

Constant 

PRUW-90 


-1.4017210 

0.006307510 


-0.94444166 

0.006826114 


-1.561182 

0.005502330 


-2.443562 

1.597810* 


-1.954103 

2.014881** 


-2.541660 

1.288903* 


*  Significant  at  the  0.10  level 

**  Significant  at  the  0.05  level 
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3.  Crew  Manning  Variables 


Variables  which  reflect  crew  manning  also  play  a  role 
in  influencing  exam  scores.  The  percent  of  turnover  in 
enlisted  personnel  in  the  engineering  department  in  the 
quarter  prior  to  the  month  of  the  exam  is  the  most  important 
factor  for  OPPEs  (PNE3-90) .  As  shown  in  Table  6,  the  signs 
of  the  estimated  slope  coefficients  are  negative  as  expected, 
TABLE  6.  OPPE  vs.  PNE3-90  (Degrees  of  Freedom  =  277) 


Parameter 


Below  Average 

Constant 

PNE3-90 

Average 

Constant 

PNE3-90 

Above  Average 

Constant 

PNE3-90 


Estimate 


-0.2919169 

-0.005819893 


0.6977273 

-0.01693883 


-1.063960 

-0.01160226 


T-Statistic 


-0.8072801 

-0.5342568 


2.295294 

-1.783363* 


-2.117330 
-0. 7316880 


*  Significant  at  the  0.05  level 


implying  that  an  increase  in  the  rate  of  personnel  turnover 
in  the  engineering  department  decreases  the  log  odds  of 
receiving  that  score.  Only  the  estimated  slope  coefficient 
for  an  average  test  score  is  significant  at  the  0.05  level. 

Variables  which  describe  the  crew  manning  for  the 
months  prior  to  an  outchop  OPPE  produce  some  interesting 
results.  Both  the  percent  of  new  engineering  department 
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personnel  in  the  quarter  prior  to  the  month  of  the  exam  (PNE3- 
90)  ,  and  the  level  of  manning  of  midc-ade  petty  officers 
shipwide  for  the  quarter  prior  to  the  exam  (M46R-90)  impact 
on  outchop  OPPE  scores.  They  are  both  presented  here  to 
indicate  their  relatively  equivalent  level  of  significance  and 
to  justify  the  variable  chosen  for  incorporation  in  the 
multivariate  outchop  OPPE  model. 

The  PNE3-90  has  a  significant  impact  on  the  scores  as 
shown  in  Table  7;  however,  the  sign  is  the  opposite  of  that 


TABLE  7.  outchop  OPPE  vs.  PNE3-90  (Degrees  of  Freedom  =  71) 


Parameter 

Estimate 

T-Statistic 

Below  Average 
Constant 

-2.818458 

-2.522146 

PNE3-90 

0.08362997 

2.048857* 

Average 

Constant 

-1.492497 

-1.719659 

PNE3-90 

0.05407537 

1.591466* 

Above  Average 
Constant 

-3 .282754 

-2.627789 

PNE3-90 

0.09071160 

2.037764** 

★ 

Significant  at  the  0.10 

level 

★  * 

Significant  at  the  0.05 

level 

which  was  expected.  It  is  positive, 

implying  that 

an 

increased  rate  of 

personnel  turnover  prior  to  an  exam 

is 

beneficial.  This 

is  a  counterintuitive 

result.  No  known 
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cause  would  produce  this  outcome.  Like  CASREP  patterns, 
personnel  turnover  patterns  are  different  during  a  deployment. 
Shipboard  personnel  tend  to  transfer  at  a  greater  rate  before 
or  after  a  deployment  vice  during  a  deployment.  So  while  the 
values  of  the  t-statistics  are  significant  for  all  estimated 
coefficients  at  a  minimum  of  the  0.10  level,  the  resulting 
reversal  in  the  signs  of  the  estimated  slope  coefficients  make 
PNE3-90  inappropriate  for  consideration  for  the  multivariate 
model . 

The  M46R-90  also  plays  a  significant  role  in 
influencing  the  scores  for  outchop  OPPEs  as  shown  in  Table  8. 
The  signs  of  the  estimated  coefficients  are  positive  as 


TABLE  8.  OUTCHOP  OPPE  VS.  M46R-90 
(Degrees  of  Freedom  =  71) 


Parameter 


Estimate 


T-Statistic 


Below  Average 

Constant  -9.7692376 

M46R-90  0.03492278 


-2.611423 

2.468589** 


Average 

Constant  -5.330087 

M46R-90  0.02021113 


-1.881447 

1.825406* 


Above  Average 

Constant  -7.000989 

M46R-90  0.02357663 


-1.872073 

1.639826* 


*  Significant  at  the  0.05  level 
**  Significant  at  the  0.025  level 


anticipated,  implying  that  a  higher  level  of  manning  is 
better.  The  t-statistic  values  indicate  that  all  the 
estimated  coefficients  are  significantly  different  from  zero 
at  less  than  the  0.05  level.  Since  this  measure  of  manning 
influence  is  more  in  line  with  expectations,  and  since  no 
apparent  explanation  for  the  sign  reversal  can  be  found  for 
the  percent  turnover  measure,  the  former  is  chosen  for 
inclusion  in  the  multivariate  model. 

Personnel  manning  also  affects  the  outcome  of  LOEs. 
Both  the  amount  of  engineering  crew  turnover  in  the  six  months 
prior  to  the  month  of  the  exam  (PNE6-180)  and  the  level  of 
manning  shipwide  for  the  junior  enlisted  personnel,  E1-E3,  for 
the  month  prior  to  the  month  of  the  exam  (M13R-90)  impact  LOE 
scores.  They  are  both  presented  in  discussion  here  to  review 
their  relative  level  of  significance  and  to  justify  the  one 
selected  for  inclusion  in  the  multivariate  model. 

Table  9  displays  the  effects  of  PNE6-180  on  LOE 
scores.  The  sign  of  the  estimated  coefficient  is  negative,  as 
anticipated,  implying  that  a  higher  rate  of  crew  turnover 
decreases  the  odds  of  receiving  a  passing  score. 

The  M13R-90  also  has  an  impact  on  the  log  odds  of 
achieving  a  passing  LOE  score,  as  shown  in  Table  10.  The  sign 
of  the  estimated  slope  coefficient  is  positive,  implying  that 
a  higher  rate  of  manning  is  better.  It  is  interesting  to  note 
that  the  t-statistic  value  is  greater  for  M13R-90  than  for 
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TABLE  9.  LOE  vs.  PME6-180  (Degrees  of  Freedom  =  96) 


PNE6-180,  even  though  the  input  variable  is  only  for  the  month 
prior  to  the  month  of  the  exam  vice  the  six  months  prior.  The 
significance  level  for  M13R-90  is  less  than  0.025.  The 
absolute  magnitude  of  the  estimated  slope  coefficient  is  also 
much  greater  for  M13R-90  than  for  PNE6-180,  even  though  the 
latter  measures  the  rate  of  crew  turnover  specifically  in  the 
engineering  department.  For  these  reasons,  M13R-30  is  the 
measure  incorporated  into  the  multivariate  LOE  model. 
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B .  MULTIVARIATE  MODELS 

Multivariate  models  are  formed  using  the  most  significant 
variables  from  the  univariate  runs.  The  goal  is  to  use  those 
measures  which  have  already  demonstrated  a  relationship  to 
capture  the  interactions  which  may  occur  among  the  input 
variables.  These  attempts  met  with  little  success.  The 
multivariate  model  for  the  OPPE  uses  DF34-90,  PRUW-180,  and 
PNE3-90,  in  Table  11.  Like  the  univariate  models,  the 


TABLE  11.  OPPE  VS.  DF34-90/PRUW-180/PNE3-90 
(Degrees  of  Freedom  =  134) 


Parameter 

Estimate 

T-Statistic 

Below  Average 

Constant 

-1.459331 

-1.873792 

DF34-90 

-0.0009838671 

-0.1187338 

PRUW-180 

0.006435732 

1.876532* 

PNE3-90 

0.0003021655 

0.2523762 

r  Average 

Constant 

-0.7600887 

-1.044490 

DF34-90 

-0.001248508 

-0.1738605 

PRUW-180 

0.008181062 

2.744023*** 

PNE3-90 

-0.01382492 

-1.111305 

I  Above  Average 

Constant 

-5.146638 

-3 . 563135 

DF34-90 

0.02929889 

1.885787* 

PRUW-180 

0.01007018 

2 . 183477** 

PNE3-90 

0.0000682056 

0.04564056 

* 

Significant  at  the 

0.05  level 

** 

Significant  at  the 

0.025  level 

★  *  * 

Significant  at  the 

0.01  level 
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value  is  so  low  as  to  make  them  insignificant.  The  importance 
of  percent  time  underway  is  underscored  by  this  model.  In 
fact,  it  clearly  dominates  the  other  input  factors.  Yet,  the 
estimated  slope  coefficients  are  not  very  great  in  magnitude. 

Attempts  to  form  a  significant  multivariate  model  for 
outchop  OPPEs  met  with  similar  results  as  shown  in  Table  12. 
The  model  uses  BC4-180,  PRUW-180,  and  M46R-90  as  independent 
variables.  Like  the  univariate  manpower  model  for  outchop 
OPPEs  which  also  uses  M46R-90,  the  significance  of  the 
estimated  slope  coefficients  for  the  manpower  measures  are  low 
with  the  exception  of  the  below  average  score.  The  signs  of 
the  estimated  coefficients  are  positive.  This  implies  that 
an  increase  in  the  level  of  manning  has  a  positive,  albeit 
small,  impact  on  below  average  exam  scores.  Also,  note  the 
relative  insignificance  of  the  percent  time  underway  as  an 
input  factor.  This  is  the  opposite  of  the  regular  OPPE,  where 
it  was  important.  Perhaps  this  is  a  result  of  the  outchop 
OPPE  being  given  at  the  end  of  a  deployment  where  all  ships 
would  normally  have  a  higher  operating  tempo,  reducing  the 
impact  of  individual  differences  on  exam  scores.  The 
magnitude  of  the  estimated  coefficients  is  higher  than 
previous  models  for  BC4-180,  indicating,  as  in  the  univariate 
outchop  OPPE  model,  that  ships  which  file  more  casualty 
reports  in  the  six  months  prior  to  the  exam  increase  the  log 
odds  of  achieving  a  passing  score. 
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i 

TABLE  12.:  OUTCHOP  OPPE  vs.  BC4-180/PRUW 
(Degrees  of  Freedom  =  47) 

-180/M46R-90 

i 

Parameter 

Estimate 

P-Value 

Below  Average 

Constant 

-13.37920 

-1.864744 

BC4-180 

0.4134871 

1.946976* 

PRUW-180 

-0.002257835 

-0.3016684 

M46R-90 

0.04615580 

1.779415* 

Average 

Constant 

-4.081422 

-0.7678036 

BC4-180 

0.2322206 

1.229982 

PRUW-180 

0.004349979 

0.6688791 

M46R-90 

0.007969758 

0.3853083 

Above  Average 

Constant 

0.2036105 

0.0325641 

BC4-180 

0.3560169 

1.674598* 

PRUW-180 

-0.009437287 

-1.141737 

M46R-90 

0.00463012 

0.1889376 

V 

★ 

Significant  at  the  0.05 

level 

The  multivariate  LOE  model  met  with  poor  results  as  shown 
in  Table  13.  It  uses  CD34-180  and  M13R-30  as  independent 
variables . 


TABLE  13.  LOE  VS.  CD34-180/M13R-90 
(Degrees  of  Freedom  =  96) 


Parameter 

Estimate 

T-Statistic 

Passing 

Constant 

0.7387126 

0.7810106 

CD34-180 

-0.002938007 

-0.3680441 

M13R-30 

0.001935358 

0.5478099 

4: 


Again,  the  motivation  for  the  selection  of  these  variables 
and  their  respective  times  rests  with  the  univariate  model. 
The  model  calculates  the  signs  of  the  estimated  slope 
coefficients  as  negative  and  positive  respectively.  This 
implies  that  fewer  C3/C4  CASREPs  and  junior  enlisted  higher 
manning  are  better.  Yet,  the  value  of  the  t-statistics  are 
so  low  as  to  make  them  insignificant. 
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VI.  CONCLUSIONS 


This  thesis  set  out  to  confirm  relationships  between  input 
measures  of  manpower,  underway  time,  and  material  readiness, 
and  their  impact  on  LOE  and  OPPE  exam  scores.  By  broadening 
the  measure  of  performance  to  reflect  the  level  of  detail 
recorded  by  the  PEB,  the  models  attempt  to  capture  these 
effects  via  multinomial  logistic  regression  using  a  software 
package  called  TSP.  These  efforts  met  with  limited  success. 

The  analysis  effort  involved  using  independent  variables 
from  the  CNA  ship  database.  The  majority  of  ships  considered 
had  complete  manpower  variables  a  result  of  past  CNA  study 
efforts.  However,  other  independent  variable  categories,  such 
as  material  and  underway  time  measures,  are  complete  only  for 
combatant  ships  such  as  cruisers,  destroyers,  and  frigate. 
This  shortfall  immediately  restricts  the  scope  of  ships  to 
which  any  possible  conclusions  may  apply.  The  database  could 
be  improved  by  incorporating  the  missing  values  for  other  ship 
types . 

The  percent  of  time  that  a  ship  spends  underway  was  the 
most  significant  of  any  independent  variable  considered.  The 
models  presented  demonstrated  for  the  OPPE  that  an  increase 
in  the  amount  of  time  that  a  ship  spends  underway  influences 
the  log  odds  of  achieving  a  desired  test  score  in  a  positive 
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way.  However,  it  would  be  inappropriate  for  decisions  makers 
to  opt  to  keep  ships  underway  all  the  time  in  an  attempt  to 
get  an  above  average  score.  Clearly,  a  ship  must  spend  some 
amount  of  time  pierside  for  maintenance  and  crew  rest.  This 
concept  implies  that  the  effects  of  underway  time  should  be 
a  quadratic  instead  of  a  linear  function  (i.e.,  more  underway 
time  is  desirable  only  to  some  point  and  then  the  ships 
experience  diminishing  returns) .  Attempts  to  fit  a  quadratic 
function  in  support  of  this  idea  failed.  Future  work  should 
focus  in  this  area  since  it  remains  an  important  issue. 
Additionally,  the  models  failed  to  show  a  strong  relationship 
between  outchop  OPPEs  and  underway  time,  perhaps  due  to  the 
fact  that  these  ships  spend  a  higher  percentage  of  time 
underway  as  a  group  resulting  in  smaller  differences  in  amount 
of  underway  times  among  ships. 

Material  measures  affected  the  exam  scores,  but  to  a 
lesser  degree.  The  most  interesting  discovery  was  that  the 
number  of  C4  CASREPs  which  begin  in  the  six  months  prior  to 
an  outchop  OPPE  has  a  positive  effect  on  achieving  a  desired 
score.  Again,  perhaps,  this  is  a  result  of  an  attempt  by 
ship's  force  personnel  to  prompt  the  supply  system  to  expedite 
required  parts.  Material  measures  have  even  less  impact  on 
LOE  scores,  due  in  part  to  the  fact  that  when  a  ship  enters 
an  extended  maintenance  period,  the  casualty  reporting  rules 
are  relaxed. 
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As  previously  noted,  CASREP  variables,  as  they  appear  in 
the  database,  reflect  degradations  throughout  the  entire  ship. 
Consequently,  equipment  failures  in  departments  other  than 
engineering  inflate  all  the  CASREP  variables  considered. 
Since  casualty  reports  specify  the  equipment  failure  and 
cognizant  department,  the  CASREP  variables  can  be  categorized 
by  department.  CNA  already  breaks  down  manpower  variables  by 
department  in  the  database  in  an  attempt  to  measure  events 
more  precisely.  Future  work  examining  the  impact  of  material 
status  engineering  scores  would  benefit  by  having  the  same 
level  of  detail  available  for  material  measures. 

Manpower  measures  also  affected  exam  scores,  but  again  to 
a  lesser  degree  than  anticipated.  The  models  constructed 
indicate  that  a  reduced  rate  of  crew  turnover  and  a  higher 
level  of  crew  manning  is  desirable  in  the  months  prior  to  the 
exam.  In  general,  the  rate  of  crew  turnover  was  a  more 
significant  influence  than  the  level  of  manning  of  the  ship. 

Other  factors  not  examined  may  be  influencing  the  exam 
results.  For  instance,  the  type  or  age  of  the  propulsion 
plant  may  also  influence  the  exam  results.  Perhaps  the  time 
of  the  year  also  affects  the  exam  outcome.  These  and  other 
possible  input  factors  are  areas  for  future  research. 

The  focus  of  the  analysis  was  to  comfirm  existing 
hypotheses  between  various  input  factors  and  exam  scores. 
This  thesis  does  not  try  to  measure  how  well  the  models 
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presented  would  perform  in  predicting  exam  performance.  Such 
an  effort  would  require  reliance  upon  some  sort  of  correlation 
coefficient,  or  r2.  An  unambiguous,  uniformly-agreed-to  r2 
measure  has  not  been  developed  for  multinomial  logit  models. 

In  summary,  the  thesis  confirmed  relationships  between 
independent  input  factors  and  dependent  exam  scores.  The 
relationships  were  significant  to  a  lesser  degree  than 
anticipated.  The  most  prominent  was  the  positive  relationship 
between  underway  time  and  the  log  odds  of  achieving  a 
successful  test  score.  Yet,  attempts  to  capture  the  effects 
of  diminishing  marginal  returns  were  not  satisfactory. 
Further  work  is  needed  to  model  these  effects. 


« 
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